This is a brief tutorial on how to configure a secure, single-user IPython notebook server to be shared between two users, with manual synchronization of contents. IPython currently does not provide real-time, Google Docs-style live synchronization (see note below).
We will assume the host machine is a Linux or OSX machine with standard POSIX user management semantics. This may also be done with a Windows host, but some of the details of this specific tutorial will need modification. Also, the actual commands were typed on Linux, so a few command-line flags may be slightly different if you are on OSX.
Note: This tutorial is aimed at users who need/want to self-host their server, for data privacy/size reasons, computational resources requirements, etc. If you can use an external service, Cloud SageMath offers hosted IPython services with real-time live sync.
With the useradd command, make a new user called shareipython (or whatever you want to call it):
sudo useradd -m -s /bin/bash shareipython
Next, set a regular password for this user:
alpamayo[~]> sudo passwd shareipython
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
and log in as shareipython:
alpamayo[~]> su - shareipython
Password:
shareipython@alpamayo:~$
From now on, you will be logged in as shareipython. Unless you know for a fact that the system version of IPython is up to date, you may want to update IPython itself (plus any other libraries you will need):
pip install --user --upgrade ipython[all]
Next, start the IPython console once, just to verify that the basics work OK:
shareipython@alpamayo:~$ ipython
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
Type "copyright", "credits" or "license" for more information.
IPython 2.2.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: exit
shareipython@alpamayo:~$
OK, at this point, you have a working IPython, in this case version 2.2.0. Now you can proceed to creating a shared notebook server.
You can find the full details on how to secure the IPython notebook in the official documentation. This document is only a brief summary of the key points, organized for easy copy-paste.
Start by creating a special profile to run the notebook server with:
ipython profile create nbserver
which should produce output like:
shareipython@alpamayo:~$ ipython profile create nbserver
[ProfileCreate] Generating default config file: u'/home/shareipython/.ipython/profile_nbserver/ipython_config.py'
[ProfileCreate] Generating default config file: u'/home/shareipython/.ipython/profile_nbserver/ipython_qtconsole_config.py'
[ProfileCreate] Generating default config file: u'/home/shareipython/.ipython/profile_nbserver/ipython_notebook_config.py'
[ProfileCreate] Generating default config file: u'/home/shareipython/.ipython/profile_nbserver/ipython_nbconvert_config.py'
And now change to the directory where those files were created:
cd /home/shareipython/.ipython/profile_nbserver/
Note: You can always locate this directory with this command:
shareipython@alpamayo:~/.ipython/profile_nbserver$ ipython profile locate nbserver
/home/shareipython/.ipython/profile_nbserver
Next, create a self-signed SSL certificate that you will use to secure the Notebook connection. Again, at the system command line (not IPython), type:
openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout mycert.pem -out mycert.pem
You will get a bunch of prompts, you can simply hit Enter to all of them and provide empty answers, they don't matter. It will look like this:
shareipython@alpamayo:~/.ipython/profile_nbserver$ openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout mycert.pem -out mycert.pem
Generating a 1024 bit RSA private key
.......................................++++++
...........................++++++
writing new private key to 'mycert.pem'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:
State or Province Name (full name) [Some-State]:
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:
Email Address []:
Now, verify that you have a certificate file:
shareipython@alpamayo:~/.ipython/profile_nbserver$ ls *.pem
mycert.pem
Next, create a password for the notebook server (this need not be the same Unix password of the user). This should be done inside of a Python or IPython shell:
In [1]: from IPython.lib import passwd
In [2]: passwd()
Enter password:
Verify password:
Out[2]: 'sha1:9b18ae137d41:004295f4352d691c295a5ef193aaf7b54a4a2864'
Save the output above, as you'll need to paste it next in a configuration file.
You now have all the pieces to set up the server. Next, you will need to edit a file named ipython_notebook_config.py in this same directory, with content like the following. Note that if you named your user something other than shareipython you'll need to adjust the path, and the hashed password should be the string you actually created. Use the content below only as a reference. IPython has already created that file for you, and there are many more parameters. The reference below is the minimal amount of configuration needed for this particular task:
c = get_config()
# The full path to an SSL/TLS certificate file.
c.NotebookApp.certfile = u'/home/shareipython/.ipython/profile_nbserver/mycert.pem'
# Listen on all IP addresses, so it can be reached over the public internet:
c.NotebookApp.ip = '*'
# Don't open the web browser when the server starts:
c.NotebookApp.open_browser = False
# Hashed notebook password that was previously created:
c.NotebookApp.password = u'sha1:9b18ae137d41:004295f4352d691c295a5ef193aaf7b54a4a2864'
# It is a good idea to put it on a known, fixed port
c.NotebookApp.port = 8989
You must deactivate autosave, so that multiple users don't overwrite each other accidentally. For that, you must edit the file custom.js located in the static/custom subdirectory of the nbserver profile (in this case, ~/.ipython/profile_nbserver/static/custom and add at the bottom:
$([IPython.events]).on("notebook_loaded.Notebook", function () {
IPython.notebook.set_autosave_interval(0);
});
Now you should be ready to run your shared notebook server. The following assumes that:
Your network has no firewalling that will prevent external users from connecting on port 8989, which you selected above. You will need to talk to your network/system administration staff if that's not the case.
No other process is running on that port. If some other user in your organization is doing so, simply select a different port number.
Since you'll need to have the server running persistently even if you log out from your computer, you should start the server process inside a persistent terminal system, like tmux or Screen.
NOTE: From now on, you should be inside a tmux (or screen) session.
First, go to the highest-level directory that you want to have available over the notebook. That may be the home directory of the special shareipython user, or a deeper one if you prefer. The notebook server can drill down deeper in the file system, but it can not go higher (for security reasons).
In that directory, start the notebook server with the specific profile you created above:
ipython notebook --profile nbserver
and you should see something along the following lines:
shareipython@alpamayo:~$ ipython notebook --profile nbserver
2014-09-19 10:31:53.348 [NotebookApp] Using existing profile dir: u'/home/shareipython/.ipython/profile_nbserver'
2014-09-19 10:31:53.353 [NotebookApp] Using MathJax from CDN: https://cdn.mathjax.org/mathjax/latest/MathJax.js
2014-09-19 10:31:53.364 [NotebookApp] Serving notebooks from local directory: /home/shareipython
2014-09-19 10:31:53.364 [NotebookApp] 0 active kernels
2014-09-19 10:31:53.364 [NotebookApp] The IPython Notebook is running at: https://[all ip addresses on your system]:8989/
2014-09-19 10:31:53.364 [NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
You can now leave this notebook running for as long as you want.
Test it by connecting to port 8989 over the https protocol, by typing in a web browser https://IP.OF.YOUR.SERVER:8989.
WARNING FOR OSX USERS: you can not use Safari for this, as it will not connect over https with a self-signed certificate. You must use Chrome or Firefox.
Because you are using a self-signed certificate, you will see a page similar to the following. This example shows the Chrome browser, but Firefox has a similar one:
Click on the advanced link at the bottom, and proceed to the destination:
This should take you to a login screen, where you will type the password you had created above:
Once you log in, you will see a standard IPython Notebook, dashboard, with one new property: it now has a "Logout" button on the upper right hand:
Now that you have a public server, you can invite colleagues to work under this shared space. Note that everyone who logs in with that password is acting as the same system user.
The trick to using the system is that there is zero automatic synchronization of the content. So you should be face to face (or on the phone/skype/g+) talking to your colleague(s) as you work, and only one person at a time can write to the file, holding a 'write lock' on the document. When the writer is ready to hand it off to someone else, they must manually save, and tell the others, who then must refresh their browser page.
Interactive widget controls (sliders, menus, etc) don't automatically reload on page refresh, so you will need to manually re-execute any cell with a live widget that you want to use after you've reloaded the page.
It can be very useful to keep one or more 'scratch' notebooks in the shared server, where various users can quickly prototype things out while the person holding the 'write lock' works on the main notebook. Then, content can be transfered from the scratch one to the main one when ready.
If two people want to work in parallel on a single notebook, another useful trick is to use the File -> Make a copy item from the menu and create a temporary copy. That has all the material from the original, which makes it easy to make further edits in isolation from the user holding the 'write lock'. Once ready, those edits can be transfered back to the main document.
The workflow is not perfect, but it works.
When behind a proxy, especially if your system or browser is set to autodetect the proxy, the notebook web application might fail to connect to the server's websockets, and present you with a warning at startup. In this case, you need to configure your system not to use the proxy for the server's address.
For example, in Firefox, go to the Preferences panel, Advanced section, Network tab, click 'Settings...', and add the address of the notebook server to the 'No proxy for' field.